6S: Distributing Crawling and Searching Across Web Peers

نویسندگان

  • Le-Shin Wu
  • Ruj Akavipat
  • Filippo Menczer
چکیده

A collaborative peer network application called 6Search (6S) is proposed to address the scalability limitations of centralized search engines. 6S peers depend on a local adaptive routing algorithm to dynamically change the topology of the peer network and search for the best neighbors to answer their queries. We validate prototypes of the 6S network via simulations with 70− 500 model users based on actual Web crawls and find that the network topology rapidly converges from a random network to a small world network, with clusters emerging from user communities with shared interests. We finally compare the quality of the results with those obtained by centralized search engines such as Google, suggesting that 6S can draw advantages from the context and coverage of the peer collective.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

6S: A Collaborative Web Search Network

6S is a collaborative peer network application, aimed to extend the current model of centralized search engines with large numbers of autonomous, distributed, micro-search engines. Each peer within the 6S network crawls the Web in a focused way, guided by its user’s information context. This way better contextual coverage can be achieved. Each peer also acts within the network by submitting, fo...

متن کامل

Crawling and Searching the Hidden Web

OF THE DISSERTATION Crawling and Searching the Hidden Web

متن کامل

Topic-Driven Crawlers: Machine Learning Issues

Topic driven crawlers are increasingly seen as a way to address the scalability limitations of universal search engines, by distributing the crawling process across users, queries, or even client computers. The context available to such crawlers can guide the navigation of links with the goal of efficiently locating highly relevant target pages. We developed a framework to fairly evaluate topic...

متن کامل

IPTV-RM: A Resources Monitoring Architecture for P2P IPTV Systems

Resources monitoring is an important problem of the overall efficient usage and control of P2P IPTV systems. The resources of IPTV can include all distributing servers, programs and peers. Several researches have tried to address this issue, but most of them illuminated P2P traffic characterization, identification and user behavior. The main contributions of this paper are twofold. Firstly, a r...

متن کامل

Semantic Overlay Networks for Peer-to-peer Web Search

We consider a network of peers, where each peer has its own collection obtained by individually crawling the web. When designing a distributed search system for such networks, an important task is how to efficiently perform query routing, i.e., how to find the most promising peers to answer the query. However, the efficiency of those routing techniques depends heavily on the underlying network ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2005